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ABSTRACT 


Facial expression plays a major role in every aspect of human life for 
communication. It has been a boon for the research in facial emotion with the 
systems that give rise to the terminology of human-computer interaction in 
real life. Humans socially interact with each other via emotions. In this 
research paper, we have proposed an approach of building a system that 
recognizes facial emotion using a Convolutional Neural Network (CNN) which 
is one of the most popular Neural Network available. It is said to be a pattern 
recognition Neural Network. Convolutional Neural Network reduces the 
dimension for large resolution images and not losing the quality and giving a 
prediction output what's expected and capturing of the facial expressions even 
in odd angles makes it stand different from other models also i.e. it works well 
for non-frontal images. But unfortunately, CNN based detector is 
computationally heavy and is a challenge for using CNN for a video as an input. 
We willimplement a facial emotion recognition system using a Convolutional 
Neural Network using a dataset. Our system will predict the output based on 
the input given to it. This system can be useful for sentimental analysis, can be 
used for clinical practices, can be useful for getting a person's review on a 
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I. INTRODUCTION 

In this world when a human being becomes speechless due to 
some reason at that moment his facial expression 
communicates many things of his state of mind at that 
particular moment. Humans always had the innate ability to 
recognize and distinguish between faces (for ex the mother 
and a child......)[3]. Humans interact socially with the help of 
emotions, which is considered a universal language. Facial 
emotions are one of the important aspects of human 
communication that help us understand the intentions of 
others [4]. Hence, itis natural that research of facial emotion 
has been acquiring a lot of attention over the past decades 
with applications not only in perceptual and cognitive science 
but also in affective computing and computer animations 
[24]. 


Human-Computer Interaction Technology (HCIT) refers to a 
kind of technology that takes computer equipment as the 
medium, to realize the interaction between human life and 
computer systems [15]. In recent years with the rapid growth 
and progress in pattern recognition and artificial intelligence, 
and many more new emerging technologies there are many 
libraries and modules available specifically for image 
processing in neural networks. Among which one of them is 
Convolution Neural Network (CNN) best suited for facial 
emotion recognition and pattern finding. 


In this paper, we are going to go through Face Emotion 
Recognition (FER) using a Convolutional Neural network for 
a better result outcome. It is also best suited for any angle 
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emotion recognition i.e. it can recognize the face emotion at 
any odd angle. Facial Expressions are responsible for 
conveying information that is hard to perceive. It has been 
applied in the fields of distance education, public security, 
public review for certain things, and many more. Hence, 
creating a system that is capable of face detection and 
emotion recognition has become a crucial area of research. 


Il. PRELIMINARIES 

2.1. Image Processing 

In our paper, we are dealing with the concept of Image 
Processing rigorously for emotion detection from an image. 
Image Processing is generally known as a methodology of 
performing some important kind of operation on an image for 
a certain output. In our paper, during this processing, we 
have input as an image and output as a characteristic/ 
features associated with that image in the form of various 
emotions such as anger, happiness, sadness, neutral, etc. that 
are expressed by human beings. 


2.1.1. Neural Network 

Neural Networks are similar to neurons in the human brain 
and works with concepts of weights, input, activation 
function, channel, etc. as follows: 

The set of elements represented by the set of input(x) in the 
form of x1,x2,x3....xn are connected to ut the connection 
between the input activation function is drawn by the set of 
weights()which are represented by 1,2,3.....n. Besides this, we 
have Bayer's letter ‘b’. 
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After applying the output of the activation function 
represented by (z) which is: y=(i wi*xi +b) 


where xi is input, i is the weight of xi, a is the bias added in 
this neuron, and is the activation function. 


This is where we collect the summation of input as a single 
unit. The other concept that is to be done is convolution 
operation convolution in images. We have single input in an 
image and we apply filters to it. 


2.1.2. Activation Function 

Generally, the Activation Function is used for determination 
purposes i.e. the output ofa neural network. There are quitea 
lot of commonly used Activation Functions, whereas in our 
case depending upon the sum of weighted input either it is 
larger than zero or not it acts as a filter deciding whether the 
information will be passed on and how strong the signal will 
be. On the other hand, if the value is equal to or less than zero 
then the signal will vanish. 


2.1.3. Stride 

Stride denotes the number of steps to be taken during the 
traversal of an array during the Pooling. The default value of 
stride is generally 1 [16]. 


2.2. The algorithm that can be used 

2.1.1. CNN 

Among the several deep learning models available, 
Convolutional Neural Network (CNN), in particular, deep 
learning is one of the most popular neural networks [4]. 
Convolutional Neural Network is one of the Feed-Forward 
Network to process and recognize image data with the grid 
version. Being one of the Neural Network it also consists of an 
Input layer where the image is input, Hidden layers such as 
Convolution Layer, ReLu Layer, Pooling Layer, and Fully 
Connected Network. 


Convolutional Layer - Converts images into an array 

The size of the feature map is controlled by three variables: 
Depth-Number of filters used for the convolution operation 
[16]. 

Stride- Number of Pixels by which the filter matrix traverses 
over the input matrix [16]. 


Padding-It’s good to input matrices with zeros around the 
bordering matrix [16]. 


Ill. RELATED WORK 

HONGLI ZHANG, ALIREZA JOLFAEI, AND MAMOUN 
ALAZAB[1].The authors have proposed a facial expression 
recognition method using the CNN model which extracts 
facial features effectively. Supervised deep learning is used as 
the processing technique in the system. The proposed 
method can automatically learn pattern features and reduce 
the incompleteness caused by artificial design features. The 
proposed method directly inputs the image pixel value 
through training sample image data. The algorithm used is R- 
CNN. 


Accuracy: 85% 

The Datasets used are: 

1. Fer-2013 facial expression database 
2. LFW data set. 


Advantages: Highly Accurate 


Limitations: Facial expressions, in reality, may vary 
depending upon various factors that may occur during image 


capturing such as face posture it can be blurring, occlusion 
may occur. It can have different noise and many more. 


Zeynab Rzayeva and Emin Alasgarov[2]. The authors have 
proposed a model based on Convolutional Neural Networks 
(CNN), which is trained on Cohn-Kanade and RAVDESS 
datasets. The proposed model gives the expected outcome for 
the said dataset in detecting macro facial emotions. The 
algorithm used is RFM technique, clustering algorithms, and 
multilayer perceptron (MLP) algorithm. 


Accuracy: 95% 
The Datasets used are: Cohn-Kanade and RAVDESS datasets 


Advantages: Distributed platforms allow us to train our 
model with a higher number of images and do it efficiently 
and effectively. 


Limitations: The algorithm is not able to detect features 
explicitly. 


LehLuoh, Chih-Chang Huang, Hsues Yen Liu[3]. The 
authors have proposed a system to build an emotion 
detection system that can analyze basic facial expressions of 
humans automatically. The experiment result also shows the 
facial emotion recognition rate is about 90% for the well- 
known JAFFE database. 


Accuracy: 90% 
The Dataset used is the JAFFE dataset. 
Advantages: Highly Accurate 


Limitations: It faces problems in image processing, such as 
detection of landmarks of face and variance of color or light of 
the environment. 


IV. CONCLUSION 

After so much research and working on this proposed system 
it has been in view that recognizing the emotion of a person 
and knowing his/her state of mind is essential. The future 
scope mentioned is endless to the counting and has many 
effective results in different fields like Medical, Security and 
Surveillance, Education, Marketing of Products, etc. with 
more futuristic applications visible on the horizon especially 
in the field of security and surveillance. 
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